ILLUMINATIONS Bloom’s dichotomous key: a new tool for evaluating the cognitive difficulty of assessments

ONE OF THE MORE WIDELY USED TOOLS to both inform course design and measure expert-like skills is Bloom’s taxonomy of educational objectives for the cognitive domain (2, 13, 22). This tool divides assessment of cognitive skills into six different levels: knowledge/remember, comprehension/understand, application/apply, analysis/analyze, synthesis/create, and evaluation/evaluate (2, 6). The first two levels are generally considered to represent lower levels of mastery (lower-order cognitive skills) and the last three represent higher-order levels of mastery involving critical thinking (higher-order cognitive skills) with apply-level questions often bridging the gap between the two (e.g., Refs. 5, 8, 10, 11, 23, and 24). While Bloom’s taxonomy is widely used by science educators, learning and mastering the concepts of the cognitive domain to categorize educational materials into the six levels identified in Bloom’s taxonomy are not trivial tasks. As with any complex task, experts and novices differ in the key abilities needed to cue into and evaluate information (4, 7, 9). Across disciplines, novices are less adept at noticing salient features and meaningful patterns, recognizing the context of applicability of concepts, and using organized conceptual knowledge rather than superficial cues to guide their decisions. Newer users of Bloom’s taxonomy demonstrate similar difficulties as they work to gain expertise, leading to inconsistencies in Bloom’s ratings (1, 8, 15) (see BDK Development for examples). To help novices gain expertise in a discipline, a common educational strategy is the use of scaffolding (7, 17, 21). Scaffolding aims to control the elements of a task, allowing a novice learner to complete the easier levels of the task and build up to the more complete and complex elements of the task (17). In the context of “Blooming,” a scaffolding structure would help the rater cue into the salient and most important elements of a question relating to the skill level of the problem and aid in using those elements to categorize the specific skill being tested in the problem. A scaffolding tool therefore provides a structure with which the novice could model their identification of key elements and decision making. One such example of a scaffolding tool to use for Bloom’s taxonomy is the Biology Blooming Tool (BBT) (8). The BBT is a conventional rubric for developing and identifying biology-specific skills and questions based on Bloom’s taxonomy. Organized as a table, each column of the rubric table outlines the key skills assessed at a given Bloom’s level (starting with the lowest level, “remember”), provides examples of exam questions, and delineates the type of exam questions that can be asked at that level. Unfortunately, in our own attempt to Bloom exam questions and course materials using a modified BBT, we had difficulty getting three independent raters to consistently rate materials. Therefore, we set out to design a new Bloom’s training tool that would provide additional, specific scaffolding that directly addressed the inconsistencies among our raters and thus might lead to greater consistency among raters. Here, we present a description of the development and evaluation of that tool: Bloom’s dichotomous key (BDK).